Skip to content

test(e2e): add outside-in walkthrough of the prisma example README#472

Open
auxesis wants to merge 11 commits into
mainfrom
test/add-e2e-for-prisma-example-app
Open

test(e2e): add outside-in walkthrough of the prisma example README#472
auxesis wants to merge 11 commits into
mainfrom
test/add-e2e-for-prisma-example-app

Conversation

@auxesis
Copy link
Copy Markdown
Contributor

@auxesis auxesis commented May 20, 2026

Add an e2e test and CI workflow that runs the steps documented in the README.md for the Prisma example app.

Background:

  • The examples/prisma app ships a README with a "Run it" section documenting the commands to run the example app
  • Nothing verified the documentation was runnable
  • Because the README is the entry point for new users, any breakages were invisible until a new user follows it and gets stuck

@auxesis auxesis requested a review from a team as a code owner May 20, 2026 15:33
@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented May 20, 2026

⚠️ No Changeset found

Latest commit: d3452b1

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 20, 2026

Warning

Rate limit exceeded

@coderdan has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 8 minutes and 53 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ffd3370f-d3bf-46bf-a27a-bb45579568fe

📥 Commits

Reviewing files that changed from the base of the PR and between 2cd61ab and d3452b1.

📒 Files selected for processing (3)
  • .github/workflows/prisma-example-readme-e2e.yml
  • e2e/README.md
  • e2e/tests/prisma-example-readme.e2e.test.ts
📝 Walkthrough

Walkthrough

Adds an end-to-end Vitest that parses and executes the Prisma example README "Run it" commands (skipping stash auth login), preserves transient artifacts, asserts command exit codes, orchestrates Docker teardown, validates pnpm start stdout, and adds a GitHub Actions workflow and README entry.

Changes

Prisma README E2E Test Suite

Layer / File(s) Summary
Auth gating, StepResult, runStep
e2e/tests/prisma-example-readme.e2e.test.ts
Auth detection via CS_CLIENT_ID/CS_CLIENT_KEY or ~/.cipherstash/auth.json, StepResult shape and describeSpawnFailure formatter, and runStep wrapper around spawnSync for standardized execution and result normalization.
Transient output snapshot/restore
e2e/tests/prisma-example-readme.e2e.test.ts
Defines TRANSIENT_PATHS and snapshot/restore/wipe utilities to preserve and restore Prisma contract files and migrations across the walkthrough.
README parsing, skip and timeouts
e2e/tests/prisma-example-readme.e2e.test.ts, examples/prisma/README.md
Parses the ## Run it bash fence from examples/prisma/README.md, sanitizes commands, defines skip set (stash auth login), and selects per-command timeouts.
Test orchestration (beforeAll/afterAll) and execution
e2e/tests/prisma-example-readme.e2e.test.ts
Gated describe.skipIf suite with beforeAll that snapshots and wipes transient outputs and runs each README command recording outcomes, and afterAll that always runs docker compose down -v, restores transient outputs, and removes generated .env.
Per-step assertions
e2e/tests/prisma-example-readme.e2e.test.ts
it.each assertions that each executed README step exited with status 0; failure output formatted via describeSpawnFailure.
CI workflow
.github/workflows/prisma-example-readme-e2e.yml
New GitHub Actions workflow that triggers on push to main and PRs (path-filtered), gates on fork secrets, injects required secrets, installs Node 22 and node-gyp, runs pnpm --frozen-lockfile, builds via turbo, runs the e2e test via turbo, and best-effort tears down Postgres with docker compose down -v.
Test documentation update
e2e/README.md
Expands the "Tests covered" table to include the supply-chain security enforcement check and the Prisma README command walkthrough (skips stash auth login).

Sequence Diagram(s)

sequenceDiagram
  participant Vitest as Vitest(Test Runner)
  participant Bash as bash
  participant DockerCompose as DockerCompose
  participant Postgres as Postgres
  participant FS as Filesystem

  Vitest->>Bash: execute README command (bash -c)
  Bash->>FS: read/write contract, migrations, .env
  Bash->>DockerCompose: start/stop services
  DockerCompose->>Postgres: provide DB service
  Vitest->>DockerCompose: docker compose down -v (teardown)
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested reviewers

  • coderdan
  • freshtonic

Poem

🐰 I parsed the README, step by step,
Spawned each command with careful prep,
I saved the contracts, wiped the rest,
Docker bowed and passed the test,
A carrot cheer — CI’s now kept! 🥕

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 14.29% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: adding an end-to-end test that walks through the Prisma example README's 'Run it' section.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test/add-e2e-for-prisma-example-app

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
e2e/tests/prisma-example-readme.e2e.test.ts (1)

66-94: 💤 Low value

Remove unnecessary async keywords.

The functions snapshotTransientOutputs, restoreTransientOutputs, and wipeTransientOutputs are declared async but contain only synchronous operations (no await statements). The async keyword is unnecessary and can be removed along with the Promise<...> return types.

♻️ Simplify function signatures
-async function snapshotTransientOutputs(): Promise<string> {
+function snapshotTransientOutputs(): string {
   const snap = mkdtempSync(join(tmpdir(), 'prisma-readme-e2e-snap-'))
   // ...
 }

-async function restoreTransientOutputs(snap: string): Promise<void> {
+function restoreTransientOutputs(snap: string): void {
   for (const rel of TRANSIENT_PATHS) {
   // ...
 }

-async function wipeTransientOutputs(): Promise<void> {
+function wipeTransientOutputs(): void {
   for (const rel of TRANSIENT_PATHS) {
   // ...
 }

Then remove await from the call sites on lines 111-112, 170, if these changes are applied.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@e2e/tests/prisma-example-readme.e2e.test.ts` around lines 66 - 94, The three
functions snapshotTransientOutputs, restoreTransientOutputs, and
wipeTransientOutputs are declared async but use only synchronous fs operations;
remove the async keyword and change their signatures from
Promise<string>/Promise<void> to plain string/void (e.g., function
snapshotTransientOutputs(): string { ... }), and update all call sites that
currently await them (calls to snapshotTransientOutputs(),
restoreTransientOutputs(), wipeTransientOutputs()) to remove the unnecessary
await so they are invoked synchronously.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/prisma-example-readme-e2e.yml:
- Around line 45-46: Replace the unpinned checkout action reference `uses:
actions/checkout@v6` with a pinned commit SHA (e.g., `uses:
actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683`) and add
`persist-credentials: false` under the checkout step to disable credential
persistence; update the SHA to the latest v6.x commit as appropriate while
keeping the `actions/checkout` identifier to locate the step in the workflow.

In `@e2e/tests/prisma-example-readme.e2e.test.ts`:
- Around line 109-159: The beforeAll timeout (600_000ms) is smaller than the sum
of the per-step timeouts used in the setup steps (outcomes.cpEnv,
outcomes.dockerUp, outcomes.pnpmInstall, outcomes.pnpmEmit, outcomes.pnpmPlan,
outcomes.pnpmApply, outcomes.pnpmStart) and may be exceeded if several steps
reach their individual timeouts; increase the beforeAll timeout to 900_000ms (15
minutes) or otherwise raise it to exceed the total step timeouts so the
beforeAll wrapper around snapshotTransientOutputs() and the runStep calls has
sufficient headroom.
- Around line 28-36: The describeSpawnFailure function currently includes raw
result.stdout and result.stderr (and other StepResult fields) in logs;
redact/sanitize sensitive plaintext before joining: in describeSpawnFailure,
sanitize result.stdout and result.stderr (and include the error.message if kept)
by replacing emails (e.g., /\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b/i) and
common connection-string patterns (e.g., postgres://, mysql://, mongodb://, and
URI-like host:port credentials) with masked placeholders, or truncate long
outputs to a safe summary, then use those sanitized strings when building
lines.join('\n'); keep the function signature and other fields (result.label,
result.signal, typeof result.status) intact while ensuring no raw plaintext
sensitive data is emitted.

---

Nitpick comments:
In `@e2e/tests/prisma-example-readme.e2e.test.ts`:
- Around line 66-94: The three functions snapshotTransientOutputs,
restoreTransientOutputs, and wipeTransientOutputs are declared async but use
only synchronous fs operations; remove the async keyword and change their
signatures from Promise<string>/Promise<void> to plain string/void (e.g.,
function snapshotTransientOutputs(): string { ... }), and update all call sites
that currently await them (calls to snapshotTransientOutputs(),
restoreTransientOutputs(), wipeTransientOutputs()) to remove the unnecessary
await so they are invoked synchronously.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ce84e65c-6676-4e9d-b346-ae7004f0aff5

📥 Commits

Reviewing files that changed from the base of the PR and between c9b2831 and 1b88054.

📒 Files selected for processing (3)
  • .github/workflows/prisma-example-readme-e2e.yml
  • e2e/README.md
  • e2e/tests/prisma-example-readme.e2e.test.ts

Comment thread .github/workflows/prisma-example-readme-e2e.yml
Comment thread e2e/tests/prisma-example-readme.e2e.test.ts
Comment on lines +109 to +159
describe.skipIf(!authConfigured)('examples/prisma README "Run it" walkthrough', () => {
beforeAll(async () => {
snapDir = await snapshotTransientOutputs()
await wipeTransientOutputs()

// Step 1: cp .env.example .env
// Run via `cp` for fidelity to the README; falls back to ENOENT spawn
// error on Windows runners (we only target Linux/macOS in CI).
outcomes.cpEnv = runStep('cp .env.example .env', 'cp', ['.env.example', '.env'], {
timeoutMs: 5_000,
})

// Step 2: docker compose up -d
outcomes.dockerUp = runStep(
'docker compose up -d',
'docker',
['compose', 'up', '-d', '--wait'],
{ timeoutMs: 180_000 },
)

// Step 3: pnpm install
outcomes.pnpmInstall = runStep('pnpm install', 'pnpm', ['install'], {
timeoutMs: 180_000,
})

// Step 4: pnpm emit
outcomes.pnpmEmit = runStep('pnpm emit', 'pnpm', ['emit'], {
timeoutMs: 60_000,
})

// Step 5: pnpm migration:plan --name initial
outcomes.pnpmPlan = runStep(
'pnpm migration:plan --name initial',
'pnpm',
['migration:plan', '--name', 'initial'],
{ timeoutMs: 60_000 },
)

// Step 6: pnpm migration:apply
outcomes.pnpmApply = runStep(
'pnpm migration:apply',
'pnpm',
['migration:apply'],
{ timeoutMs: 120_000 },
)

// Step 7: pnpm start
outcomes.pnpmStart = runStep('pnpm start', 'pnpm', ['start'], {
timeoutMs: 120_000,
})
}, 600_000) // 10 min total budget for the cold path
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Potential timeout mismatch in beforeAll.

The beforeAll timeout is 600,000ms (10 minutes), but the sum of individual step timeouts is 725 seconds (~12 minutes): cp (5s) + docker up (180s) + install (180s) + emit (60s) + plan (60s) + apply (120s) + start (120s). While commands typically complete much faster than their timeout limits, if multiple steps hit their timeouts, the beforeAll could exceed its budget.

Consider increasing the beforeAll timeout to 15 minutes (900,000ms) to provide headroom, or document that the 10-minute limit assumes normal execution.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@e2e/tests/prisma-example-readme.e2e.test.ts` around lines 109 - 159, The
beforeAll timeout (600_000ms) is smaller than the sum of the per-step timeouts
used in the setup steps (outcomes.cpEnv, outcomes.dockerUp,
outcomes.pnpmInstall, outcomes.pnpmEmit, outcomes.pnpmPlan, outcomes.pnpmApply,
outcomes.pnpmStart) and may be exceeded if several steps reach their individual
timeouts; increase the beforeAll timeout to 900_000ms (15 minutes) or otherwise
raise it to exceed the total step timeouts so the beforeAll wrapper around
snapshotTransientOutputs() and the runStep calls has sufficient headroom.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
e2e/tests/prisma-example-readme.e2e.test.ts (1)

170-170: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Guard restore when snapshot creation failed early.

Line 170 assumes snapDir is always assigned. If beforeAll fails before assignment, afterAll can throw here and mask the root failure.

Suggested minimal fix
-let snapDir: string
+let snapDir: string | undefined
...
-    await restoreTransientOutputs(snapDir)
+    if (snapDir) await restoreTransientOutputs(snapDir)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@e2e/tests/prisma-example-readme.e2e.test.ts` at line 170, The afterAll
cleanup unconditionally calls restoreTransientOutputs(snapDir) which can throw
if snapDir was never assigned due to a beforeAll failure; modify the afterAll to
guard the call by checking that snapDir is defined/non-empty (or that the
snapshot was created) before invoking restoreTransientOutputs, e.g., wrap the
restoreTransientOutputs(snapDir) call in an if (snapDir) / truthy check so
failures in beforeAll are not masked by a cleanup error.
♻️ Duplicate comments (2)
e2e/tests/prisma-example-readme.e2e.test.ts (2)

145-159: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Increase beforeAll timeout to match step-budget worst case.

Line 159 keeps 600_000 ms, but configured/default step timeouts can exceed that in slow CI paths, causing wrapper timeout before step-level failures are reported.

Suggested minimal fix
-  }, 600_000) // 10 min total budget for the cold path
+  }, 900_000) // 15 min budget to cover worst-case step timeout sum
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@e2e/tests/prisma-example-readme.e2e.test.ts` around lines 145 - 159, The
beforeAll wrapper timeout (currently the fixed 600_000) can be shorter than the
sum of per-step timeouts and causes premature test termination; replace the
hardcoded 600_000 with a computed total budget based on the README_COMMANDS step
timeouts (e.g. const totalBudget = README_COMMANDS.reduce((acc, line) => acc +
timeoutFor(line), 0) + 60_000) and use totalBudget as the beforeAll timeout so
the outer hook always exceeds the worst-case step budget when running runStep
over README_COMMANDS (skip logic with SKIP_COMMANDS and outcomes map remains
unchanged).

33-34: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Redact process output before appending failure diagnostics.

Line 33 and Line 34 still include raw stderr/stdout, which can emit plaintext values in CI logs. Please sanitize (or summarize/truncate) these fields before lines.push(...).

Suggested minimal fix
+function redactPlaintext(s: string): string {
+  return s
+    .replace(/\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b/gi, '[redacted-email]')
+    .replace(/\b(?:postgres|mysql|mongodb):\/\/\S+/gi, '[redacted-conn-string]')
+    .slice(0, 4000)
+}
+
 function describeSpawnFailure(result: StepResult): string {
   const lines = [`step \`${result.label}\` failed.`]
   if (result.error) lines.push(`  spawn error: ${result.error.message}`)
   if (result.signal) lines.push(`  killed by signal: ${result.signal}`)
   if (typeof result.status === 'number') lines.push(`  exit status: ${result.status}`)
-  if (result.stderr.trim()) lines.push(`--- stderr ---\n${result.stderr.trim()}`)
-  if (result.stdout.trim()) lines.push(`--- stdout ---\n${result.stdout.trim()}`)
+  const safeStderr = redactPlaintext(result.stderr).trim()
+  const safeStdout = redactPlaintext(result.stdout).trim()
+  if (safeStderr) lines.push(`--- stderr ---\n${safeStderr}`)
+  if (safeStdout) lines.push(`--- stdout ---\n${safeStdout}`)
   return lines.join('\n')
 }

As per coding guidelines, Never log plaintext data; the library by design never logs plaintext sensitive information.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@e2e/tests/prisma-example-readme.e2e.test.ts` around lines 33 - 34, The test
currently appends raw process output (result.stderr and result.stdout) into
diagnostics via lines.push, which can leak plaintext; replace those direct
appends with sanitized or truncated summaries: for example compute a redacted
string from result.stderr and result.stdout (e.g., mask sensitive patterns,
limit length, or replace with "<redacted>"/first-N-chars + "...") and push that
sanitized string instead; update the lines.push calls that reference
result.stderr, result.stdout so they use the sanitized variables and include
contextual labels like "--- stderr ---" / "--- stdout ---" as before.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@e2e/tests/prisma-example-readme.e2e.test.ts`:
- Line 170: The afterAll cleanup unconditionally calls
restoreTransientOutputs(snapDir) which can throw if snapDir was never assigned
due to a beforeAll failure; modify the afterAll to guard the call by checking
that snapDir is defined/non-empty (or that the snapshot was created) before
invoking restoreTransientOutputs, e.g., wrap the
restoreTransientOutputs(snapDir) call in an if (snapDir) / truthy check so
failures in beforeAll are not masked by a cleanup error.

---

Duplicate comments:
In `@e2e/tests/prisma-example-readme.e2e.test.ts`:
- Around line 145-159: The beforeAll wrapper timeout (currently the fixed
600_000) can be shorter than the sum of per-step timeouts and causes premature
test termination; replace the hardcoded 600_000 with a computed total budget
based on the README_COMMANDS step timeouts (e.g. const totalBudget =
README_COMMANDS.reduce((acc, line) => acc + timeoutFor(line), 0) + 60_000) and
use totalBudget as the beforeAll timeout so the outer hook always exceeds the
worst-case step budget when running runStep over README_COMMANDS (skip logic
with SKIP_COMMANDS and outcomes map remains unchanged).
- Around line 33-34: The test currently appends raw process output
(result.stderr and result.stdout) into diagnostics via lines.push, which can
leak plaintext; replace those direct appends with sanitized or truncated
summaries: for example compute a redacted string from result.stderr and
result.stdout (e.g., mask sensitive patterns, limit length, or replace with
"<redacted>"/first-N-chars + "...") and push that sanitized string instead;
update the lines.push calls that reference result.stderr, result.stdout so they
use the sanitized variables and include contextual labels like "--- stderr ---"
/ "--- stdout ---" as before.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d899ed80-c819-4805-b175-540c6c4792e1

📥 Commits

Reviewing files that changed from the base of the PR and between 1b88054 and 41bb0d7.

📒 Files selected for processing (1)
  • e2e/tests/prisma-example-readme.e2e.test.ts

auxesis added 11 commits May 21, 2026 22:14
… steps

Replace the hardcoded list of 7 'Run it' commands in
e2e/tests/prisma-example-readme.e2e.test.ts with a parseRunItCommands
helper that reads examples/prisma/README.md at test-collection time
and executes whatever's in the bash fence under '## Run it'. The
README is now the single source of truth — if a command gets renamed
the test runs the new wording and fails clearly, instead of silently
passing on the old wording.

Per-step exit-zero assertion uses it.each over the parsed lines so
each parsed command shows up as its own row in the vitest reporter.
'stash auth login' (interactive PKCE) is skipped via an exact-match
SKIP_COMMANDS set; brittle to wording changes by design so README
drift surfaces loudly. Per-command timeouts mapped via a small
prefix table.

Side-effect assertions (.env exists, contract.{json,d.ts}, the
migrations/app/<...>_initial directory, pg_isready) and the 'Expected
output' string list remain hardcoded — they catch 'exit 0 but
produced no output' regressions and aren't parseable from the README.

Verified:
- 13/13 pass in ~12s (warm cache)
- drift simulation: renamed 'pnpm emit' -> 'pnpm emiit' in README, test
  failed with 'README "Run it" step exited 0: pnpm emiit' surfacing
  pnpm's 'Did you mean "pnpm emit"?' stderr
- working tree clean after run, container torn down
Drop the six brittle assertions beyond the per-command exit-zero
check: two that inspect pnpm start stdout for specific codec headings
and row counts, and four side-effect checks (.env exists, pg_isready,
contract.{json,d.ts} exist, migrations/app/*_initial exists).

These broke for reasons unrelated to the test's contract — any
wording change in src/index.ts tripped the stdout checks, and the
side-effect checks hardcoded the CLI's migration-naming convention,
the container name, and the emit output paths.

The test is an outside-in smoke test: it verifies every command in
the README's documented walkthrough runs successfully. It now asserts
exactly that and nothing more — parse the README, run each command,
assert exit 0.

Accepted tradeoff: a command that exits 0 but silently produces
nothing is no longer caught.

Verified: 7/7 exit-zero rows pass (~12s warm cache), test file
typechecks clean, working tree clean after run.
Since the test runs every command via `bash -c` and asserts only
exit-zero, parts of runStep's signature are vestigial:

- the cmd + args split existed for the old hardcoded
  runStep('pnpm', ['emit']) calls — every call is now bash -c <line>
- opts.cwd / opts.env were never passed — always EXAMPLE_DIR /
  process.env
- the timeoutMs default was never reached — both call sites pass an
  explicit timeout

Collapse to runStep(commandLine, timeoutMs). Drop the redundant
SpawnSyncReturns annotation (TS infers it) and its now-unused import.
afterAll's teardown runs via bash -c too, consistent with the
walkthrough — bash is already a hard dependency of every step.

Also fix the e2e/README.md coverage row, which still claimed the test
asserts pnpm start matches the documented Expected output — removed in
2cd61ab.

No behaviour change. Verified: 7/7 exit-zero rows pass (~12s warm),
typecheck clean, working tree clean after run.
@coderdan coderdan force-pushed the test/add-e2e-for-prisma-example-app branch from d809bb8 to d3452b1 Compare May 21, 2026 12:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant